Sharp MSE Bounds for Proximal Denoising

Denoising has to do with estimating a signal x0 from its noisy observations y = x0 + z. In this paper,we focus on the “structured denoising problem”, where the signal x0 possesses a certain structure and zhas independent normally distributed entries with mean zero and variance σ. We employ a structure-inducing convex function f(·) and solveminx{ 12‖y − x‖ 22 + σλf(x)} to estimate x0, for some λ > 0.Common choices for f(·) include the `1 norm for sparse vectors, the`1− `2 norm for block-sparse signalsand the nuclear norm for low rank matrices. The metric we use to evaluate the performance of an estimatex∗ is the normalized mean-squared-error NMSE(σ) = E‖x−x0‖2σ2 . We show that NMSE is maximized asσ → 0 and we find the exact worst case NMSE, which has a simple geometric interpretation: the mean-squared-distance of a standard normal vector to the λ-scaled subdifferential λ∂f(x0). When λ is optimallytuned to minimize the worst-case NMSE, our results can be related to the constrained denoising problemminf(x)≤f(x0){‖y − x‖2}. The paper also connects these results to the generalized LASSO problem, inwhich, one solvesminf(x)≤f(x0){‖y−Ax‖2} to estimate x0 from noisy linear observations y = Ax0 + z.We show that certain properties of the LASSO problem are closely related to the denoising problem. Inparticular, we characterize the normalized LASSO cost and show that it exhibits a “phase transition” asa function of number of observations. Our results are significant in two ways. First, we find a simpleformula for the performance of a general convex estimator. Secondly, we establish a connection betweenthe denoising and linear inverse problems.